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ABSTRACT 

The modified American Council on the Teaching of 
Foreign Languages Educational Testing Service (ACTFL/ETS) oral 
proficiency scale was developed in response to a request from the New 
Brunswick (Canada) Department of Education for assistance in 
comparing the results of its French immersion and core programs. 
Because the existing ACTFL/ETS academic language scale provided 
inadequate discrimination for the skills of immersion students, two 
researchers interviewed students from both the core and the immersion 
programs in grades 7 through 9 in Fredericton, New Brunswick, and 
developed a provisional rating scale, a set of sample or practice 
tapes with descriptions of the speech samples, and a set of test 
tapes for interview raters. Some of the findings were the following: 
there was no overlap of oral proficiencies between the core and 
immersion students; with the exception of pronunciation, the 
unmodified novice level descriptions applied to the core students; 
the modified ratings, labeled "junior," include some content areas 
typically rated "novice"; the differences within the relatively 
fluent immersion group were primarily in their abilities to discuss 
familiar or unfamiliar, complex situations, and concrete or abstract 
ideas. The major limiting factor was their speech accuracy. The 
junior rating scale descriptions are included. (MSE) 
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This project was initiated in 1983 by the New Brunswick, 
Canada, Department of Education who asked ETS for assistance in 
evaluating the success of one aspect, the students ' speaking 
ability, of the French immersion program available to anglophone 
school children in New Brunswick. Specifically, the Department 
of Education was interested in finding meaningful measures to 
assess the oral proficiency in French of junior high school 
students who had participated in both the immersion and core 
programs of study. 

In New Brunswick, students of English speaking families 
(anglophone) that register for the "core" program of language 
study receive instruction in French for one class period a day, 
usually starting in the first grade. Students who participate in 
the "immersion" program receive all their instruction in French 
from the first school day of first grade, even though they may 
never have been exposed to French before that time. These 
students do not learn to read English until the fifth grade. 
After the fifth grade, the number of classes in English gradually 
increases until half the classes are in English at the end of 
junior high (9th grade). 

Since the mid 1970's the province of New Brunswick had been 
using the U.S. government Foreign Service Institute (FSI) scale 
to evaluate the oral proficiency of graduating high school 
seniors in both French and English (as second languages). Not 
surprisingly, they found this scale inadequate for discriminating 
the oral proficiency in French among junior high school students 
since that scale was not developed for academic use. 

ETS's first response to the request from the New Brunswick 
Department of Education was to submit the ACTFL/ETS Academic 
^> rating scale for their consideration, since this scale provides 

^ for greater discrimination at the lower levels than does the FSI 

scale. The Department of Education did not consider the 
\t descriptions of the various levels relevant to their students and 

O asked ETS to listen to taped interviews of some of their junior 

q high school students. Minor revisions based on some of these 

{ERIC ta P es (such as changing all references to professional or 

work-related topics to school-rela^d topics) still proved to be 



unsatisfactory. The New Brunswick Department of Education 
continued to maintain that the descriptors of the ACTFL/ETS 
Academic rating scale simply did not pertain to students from 
immersion classes. For example, they claimed that the majority 
of the students could do more than "satisfy most work/school 
requirements and show some ability to communicate on concrete 
topics related to particular interests and special fields of 
competence" (Advanced level on ACTFL/ETS scale). Yet it seemed 
unreasonable to assume that all the students were at or above the 
Advanced level. Surely, there was a possibility that greater 
variation in speaking ability existed among these students. 
Furthermore, although each level of the academic rating scale is 
intended to cover a fairly broad range of oral proficiency, a 
scale that fails to discriminate among the majority of the 
population it is intended to serve is necessarily of limited 
usefulness. 

Therefore it was decided that we (Kate Rabiteau and Hessy 
Taft) would go to New Brunswick to conduct interviews for junior 
high school students in both core and immersion programs. 

THE STUDY 

In April and in June 1984, we interviewed students in grades 
7 through 9 in Fredericton, New Brunswick. In April, school 
teachers selected the students to be interviewed. In June, ETS 
staff selected students by holding screening sessions with large 
numbers of students. The screening sessions consisted of 
informal introductions on the part of the students and statements 
about their major interests. Separate screening sessions were 
held for students from "core" and "immersion" programs. Students 
were deliberately chosen to represent the broadest ranges of 
speech samples possible. 

The outcomes of the study are a provisional rating scale of 
oral proficiency applicable to junior high school students, / set 
of sample or practice tapes with descriptions of the speech 
samples, and a set of test tapes for prospective raters of the 
oral proficiency interview. The manual that accompanied these 
materials was essentially the ACTFL/ETS manual on oral 
proficiency. 

THE FINDINGS 

The results obtained from this study were most interesting. 
First of all, it became immediately apparent that there was no 
overlap of oral proficiencies between the students in core and 
immersion programs. In fact we found that students fell into a 
bi-modal distribution whereby students from core programs rarely 
attained a rating higher than the equivalent of Novice-High and 
students from immersion programs rarely attained ratings below 
the equivalent of Intermediate-High. 
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Secondiy, we found that, with one exception, the ACTFL/ETS 
descriptions for the Novice levels were quite applicable to 
rating the speech samples of students from the core programs. 
This is not surprising at the lowest levels of the scale. The 
one exception is the reference to pronunciation which generally 
ceases to be a matter of much concern when dealing with students 
exposed to French in the early grades. By and large, these 
students are intelligible even to native speakers not used to 
dealing with foreigners. This being the case, references to 
intonation and articulation were not included in the rating scale 
that was developed for junior students. 

On the other hand, the revised rating scale, which 
provisionally consists of the ACTFL/ETS labels preceeded by the 
word "Junior", includes in the descriptors some content areas 
typically associated with Hovice-level speech. Since students at 
the Novice level do not create with language but rely on learned 
materials, it seemed appropriate to us to include typical low 
level content areas in the descriptors. 

The situation for students from the immersion programs was 
quite revealing. Most oi the 12-, 13-, or 14-year olds we 
interviewed had remarkable fluency, being able to process speech 
sufficiently well to produce utterances of considerable lengths, 
indeed to sustain dialogue and show considerable ability to 
create with language. Students in this group were generally 
willing, indeed eager, to provide description and narration. 
Many even showed little hesitancy to contradict or to present an 
argument. The differences among students from this group were 
evident primarily in their relative abilities to discuss familiar 
and unfamiliar topics, to deal with complicated or unfamiliar 
situations, and to discuss concrete or abstract ideas. 

The major limiting factor among these students was the 
accuracy of their speech, which trailed considerably behind their 
ability to function in the language generally or to tackle 
content/context issues appropriately. For example, students 
would typically be able to communicate their thoughts abou^ 
' current, past and future activities but errors in both formation 
and selection of tense forms were frequent. On the content 
level, a student might attempt to explain why the invasion of 
Normandy in World War II was necessary or venture an opinion on 
whether military service should be required for everyone, both 
examples of higher level topics. Neither of the students who 
responded to these questions were actually rated as Superior 
because they did not respond satisfactorily to the probes: they 
could not sustain discourse at that level and their ability to 
handle the accuracy of the language was much too limited. These 
are unusual topics for 13-14 year olds to discuss even in their 
native language but we did encounter some students who could 
respond appropri :;cely on such content /context areas. This 
observation is supported by repeated evidence of extensive or low 
frequency vocabulary. However, for the majority of immersion 
students there were significant gaps in their mastery of the 

ER?C . 



-4- 



morphological system of the language. Although most students had 
little trouble joining sentences for general discourse, the lack 
of control of syntactic structures, (some simple, some complex) 
was widespread. Errors in basic structures such as "mon frere, 
il est 6 ans" or "j'ai rest^, j'ai alle" were common and appeared 
to have formed a pattern. (On the other hand, "j'ai faim" or 
"j'ai froid" were commonly used correctly with no interference 
from English in these instances). For most immersion students, 
the errors in their speech generally did not interfere with 
understanding. However, in the case of weaker students in this 
group, errors did occasionally inhibit the compreheneion of the 
message. 

It is not clear why some of these grammar errors persist. 
Perhaps teachers choose to deemphasize them in an attempt not to 
inhibit speech in young speakers. Much research is needed on the 
subject, particularly on the sequence of acquisition of language 
skills in young learners. 

Heidi Byrnes notes in an article recently submitted for 
publication that "increasing evidence from immersion projects.... 
indicates that naturalistic acquirers in those surroundings fall 
short on a wide variety of morpho- syntactic constructions and, in 
fact, seem to be fossilizing in their language use". Byrnes 
suggests that formaX language instruction may serve to inhibit 
"the use of ungrammatical though communicatively effective 
constructions". The introduction and sequencing of such 
instruction in the case of young learners is a recurrent theme 
deserving attention. As Barnes states, "What is necessary is 
evidence from a longitudinal study controlling for the various 
types of input, naturalistic or instructed, and then comparing 
the language use of the respective groups". 

In the present study, it seemed reasonable that the rating 
scale for the intermediate and advanced levels for juniors would 
be more applicable to junior students if the function criteria 
were made somewhat more stringent than those described in the 
ACTFL/ETS academic rating scale and the accuracy criteria were 
made somewhat more lenient. The descriptions were modified to 
incorporate these shifts. 

One of the major features in which the junior scale differs 
from the ACTFL/ETS academic rating scale is that \:ne junior scale 
does not require evidence of firm control of both past and future 
time to be rated at the Advanced level. We woulv* like to point 
out that if this were not the case, the overwhelming majority of 



The second paragraphs of the descriptions for Advanced 
and Superior levels pertain to aspects of grammar specifically 
applicable to French and can be deleted for more general 
nonl an guage- specif ic descriptions. 

O 
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the students in the immersion programs would fall into the 
Intermediate High category. 

We should also point out that the "areas of weakness" 
regarding accuracy in the Advanced Plus level of the ACTFL/ETS 
academic rating scale are probably intended to refer to the 
speech of a testee under stress. Repeated evidence of such areas 
of weakness would not, in practice, merit an Advanced Plus 
rating. 

In the junior scale under consideration, we have placed such 
areas of weakness in the Jr. Advanced level rather than Advanced 
Plus level because stress does not appear to be a major factor 
inhibiting a testee's speech at that level. While at first 
glance, it may appear that we are setting more stringent 
standards for accuracy than the ACTFL/ETS scale, in fact this is 
not the case. The "areas of weakness" in question can be 
expected to be routine at the Jr. Advanced level. This practice 
is consistent with a more lenient approach to accuracy for junior 
students . 

An example of the more stringent requirements for 
functioning in the language is found in the ability to handle 
complicated situations. The junior scale at the Advanced level 
expects students to have "some facility in handling complications 
or difficulties" and in particular be able to communicate 
effectively complications arising from a familiar situation (such 
as calling for help when an accident has occurred). Students at 
this level are expected to be adept at problem solving. The 
ACTFL/ETS Advanced level requires similar capabilities in 
practice although they are only indirectly implied in the 
description itself. 

A typical example of a student at the Advanced level on the 
junior rating scale would be one who can respond to a 
hypothetical situation with relevant vocabulary and appropriately 
connected sequences or utterances but with relatively weak 
control of certain grammatical structures. 

The junior level academic rating scale also contains a 
description of a Superior level performance. Sine** no 
modifications are required when dealing with speakers capable of 
expressing abstract thought, the description is, in fact, no 
different from the ACTFL/ETS description. (It is therefore not 
prefaced by the word "junior"). The Superior level has been 
described here because a few of the speakers were found to be at 
that level. There was thus a clear need to illustrate the key 
features that characterize the higher levels of speech and to 
document in the description some of the valuable information 
provided at ACTFL/ETS led oral proficiency workshops. The 
revised junior academic scale is truncated at the Superior Plus 
level. 
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CONOT.tlSTON: 



The immediate goal of this project was to adapt the 
ACTFL/ETS rating scale to the situation of second language 
students in New Brunswick elementary and junior high schools, 
The long range goal that was foremost in our considerations was 
to write descriptions for a junior academic rating scale that 
would have wide applicability across languages for young learners 
in a wide variety of second language programs. Because it was 
apparent that the sequence of language acquisition among young 
learners in immersion programs appears not to be the same as that 
described in the academic ACTFL/ETS scale, different emphases 
were placed on the components of the functional trisection of the 
oral proficiency rating scale. One should note that the oral 
proficiency of young learners in more traditional programs such 
as the core program described in this study was confined largely 
to the Notice levels and conformed closely to the corresponding 
ACTFL/ETS descriptions. 

In any modification of the rating scale, it should be of 
paramount importance to make the descriptions sufficiently 
distinct from each other to permit high rater reliability among 
users while maintaining the broad characteristics of language 
acquisition of the intended population. It would be of interest 
to ascertain whether the findings obtained in this study can be 
corroborated by further investigation of the speaking performance 
of students in other immersion programs. 
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ACADEMIC ORAL PROFICIENCY RATING SCALE 
(Jr. High School and Above) 

Rating Description 

0 No ability whatsoever in the language. 



Jr. Novice Unable to function in the spoken language 

Low 

Or^l production is limited to occasional isolated words 
or expressions that are cognates of English words* Essentially 
no ability tc communicate. 



Jr. Novice Able to operate only in a very limited capacity within very 

Mid predictable areas of need . Vocabulary limited to that necessary 

to express basic courtesy formulae and to identify components in 
several of the following subject areas: basic objects, colors, 
clothing, numbers, family members, food, months, time, weather, 
weekdays, year. Syntax is fragmented, inflections and word endings 
frequently omitted, confused or distorted and the majority of 
utterances consist of isolated words or short phrases. Speakers at 
this level do not show evidence of creating with language or being 
able to ccpe with simple situations. Speech is limited to 
phrases rather than complete sentences and is marked by frequent 
long pauses. Repetition of interlocuter's words is often 
necessary. 



Jr. Novice Able to satisfy immediate needs using learned utterances . 

High There is no real autonomy of expression, although there are some 

emerging signs of spontaneity and flexibilty. There is a slight 
increase in utterance length but frequent long pauses still occur. 
Comprehension is often limited enough to inhibit normal conversation. 
Can ask questions or make statements with reasonable accuracy only 
where this involves short memorized utterances iorJ:ormulae. Can 
handle with confidence vocabulary related to JSy&f the following 
subject areas: basic objects, colors, clothing, numbers, family 
members, food, months, time, weather, weekdays, year. Can handle 
general high frequency utterances. Is sometimes but not con- 
sistently able to remember learned elements to produce simple 
original se *ences. At times, can handle simple situations but 
cannot sustain performance that demonstrates ability to create with 
language. Usually unable to make one's needs known and communicate 
essential information in a simple survival situation. There is 
some concept of the present tense forms of the verbs and some 
common irregular verbs, although use is limited primarily to first 
person singular and first and second person plural. There is 
some use of articles, indicating a concept of gender, although 
mistakes are numerous. 
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Rating 

Jr. Inter- 
mediate Low 



Description 

Able to satisfy bejic survival needs and minimum courtesy 
requirements . In areas of immediate need or on very familiar topics, 
can ask and answer simple questions, initiate and respond to 
simple statements, and maintain very simple face-to-face con- 
versations. When asked to do so, is able to formulate simple 
questions. Most utterances are limited to simple sentences and 
contain fractured syntax or other grammatical errors. Vocabulary 
adequate to express basic elementary needs such as ordering a 
meal or asking for directions, and to handle simple exchanges 
of information pertaining to <>Aily activities. Misunderstandings 
may still arise due to limited comprehension. There is some 
control of the present tense of regular verbs and the more common 
irregular verbs and of gender, number, and subject-verb agree- 
ment. Can give simple answers In the negative. Little precision 
in information conveyed owing to tentative state of grammatical 
development and little or no use of modifiers. 



Jr. Inter- Able to show some spontaneity in language production and to 

mediate Mid initiate and sustain simple dialogue . Vocabulary permits dis- 

cussion of topics beyond basic survival needs. Can use 
language creatively can maintain simple face-to-face conversation. 
Can talk simply about autobiographical details, leisure time 
activies and daily schedule. Comprehension good enough so as not 
to interfere with normal conversation. Some grammatical accuracy 
in basic constructions, i.e., subject-verb agreement, noun- 
adjective and gender agreement for familiar vocabulary, present 
tense of regular verbs and common irregular verbs. May have 
a concept of past time, but can use only isolated past tense forms 
which have been learned as vocabulary items. Syntax in most 
simple declarative sentences is generally correct, including 
placement of most common adjectives. 



Jr. Inter- 
mediate High 



Able to show considerable spontaneity in language production 
and t o initiate and sustain general conversation . Developing 
flexibility in a range of wircumstances beyond immediate survival 
needs. Can use language creatively to initiate and sustain 
routine conversation but accuracy may be uneven. Ability to 
describe and give precise information is limited to present 
tense. Other tense forms occur although errors in formation aid 
selection are frequent. Is able to produce some narration in 
present, past and/or future time but cannot sustain performance 
at this level. Can use most question forms. While some word order 
is established, errors still occur in mor<s complex patterns. Speaking 
performance Is often uneven (strong in either grammar or vocabulary, 
but not both, for example) but shows remarkable fluency. Cannot 
sustain coherent structures in longer utterances by the use of 
conjunctions or relative clauses. Some control of past tense forms 
and basic reflexive verbs. May be able to use some direct and in- 
direct object pronouns, although syntax may still be faulty. 



Rating Description 

Jr. Advanced Able to satisfy routine vork or school requirements and to communi- 
cate in a limited manner on concrete topics relating to particular 
interests and special fields of competence , Can handle with confi- 
dence most social situations related to vork or school requirements 
including some facility in handling complications or difficulties 
arising from these. Has a speaking vocabulary sufficient to 
respond, with circumlocutions, on concrete topics relating to 
particular Interests and special fields of competence. Can narrate, 
describe, and explain in present, past, and future time although 
errors still occur. Can communicate facts — what, who, when, where, 
how much — and can explain points of view in an uncomplicated 
fashion, but cannot conjecture or coherently support an opinion. Can 
talk in a general way about topics of current public interest (e.g., 
student rules and regulations), as well as personal interest. Can 
make factual comparisons, such as high school life vs. college life 
or elementary school life. Can communicate needs and thoughts in a 
familiar situation with a complication (e.g., calling for help when 
an accident has occurred). Speaking performance shows high degree of 
fluency and ease of speech. 

Areas of weakness in grammar can range from simple constructions 
such as noun-adjective, gender, and subject-verb agreement to more 
complex structures such as tense usage (Imperfect vs. pass£ compose), 
and relative clauses. Is able to use the partitive (affirmative 
and negative), demonstrative adjectives, most expressions of quantity, 
most adverbs, and some idiomatic expressions. 

Ji . Advanced Able to satisfy most work or school requirements and to coimunicate 
Plus on most concrete topics relating to particular interests and special 

fields of competence . Can handle with confidence work or school re- 
quirements involving complications that arise in daily life. Can 
talk about facts related to topics of social importance or current 
interests. Generally quite adept in using circumlocutions in 
unfamiliar situations dealing with concrete topics. Can occasionally 
give supported opinions or respond to hypothetical situations but 
responses are often fractured and limited to simple structures. Can 
narrate and describe comfortably in present, past, and future time. 
Vocabulary is broad enough for fluent speech although searching for 
every day words may still be evident. Can usually handle elementary 
constructions quite accurately, such as the present, past, future tenses 
of regular and irregular verbs. Has good control of the reflexive 
and impersonal form of verbs and of imperatives. However, use of 
conditional sentences is unstable. Can link sentences together 
by using conjunctions and subordinate clauses. 

Is able to use adjectives (including tout, comparative and superlative 
forms) correctly, and can handle object pronouns, interrogative 
pronouns, relative pronouns, negative patterns other than ne ... pas, 
most prepositions and some idioms * ic expressions. Generally, syntax 
is correct and word order is sustained with all pronouns, including y 
and en* Grasps but does not control the basic differences in usage 
between the pass! compose and the imperfect (repeated or continuous 
vs. single action in the past, etc.) so that mistakes are to be 
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Ra ting Description 

Superior Able to speak the language with sufficient structural accuracy 

and v ocabulary to participate effectively in most formal and informal 
conversations on practical, social, and professional topics . 
Can discuss particular interests, special fields of competence, current 
events and societal problems effectively by supporting opinions in 
thorough manner. Can respond with considerable ease tc hypothetial 
situations and discussions of abstract concepts. Vocabulary is 
extensive and groping for words is rare. Shows familiarity with 
idiomatic expressions and facility with lower frequency utterances 
that permit discussion of topics in unfamiliar situations. Control 
of grammar strong enough so that use of structures and syntax reveal 
no major patterns of error in basic constructions. Able to argue 
and debate a point of view. 

Good control of pass/' compost' vs. imperfect; uses the present 
subjunctive appropriately most of the time. Shows control 
of si clauses, particularly in hypothetical statements. Correct 
use of the future tense with conjunctions such as quand, des que, 
etc., and good control of negations, including ne. . .ni. . .ni, 
negation of the infinitive, and ne...que. 



Superior (Not applicable to junior students) 

Plus ALL PERFORMANCE ABOVE SUPERIOR IS RATED SUPERIOR PLUS. 

(Corresponds to Level 3+ or higher in U.S. Government 
Defense Language Institute Rating Scale.) 

Performance is characterized by awareness of sociolinguistic cultural 
factors, some facility with colloquialisms * ability to counsel, 
persuade, and negotiate convincingly in the language, and ability 
to tailor speech to the target audience; errors of pronounciation 
and grammar are quite rare; can handle informal interpreting 
from and into the language. 
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